A Multimethod Usability Approach to Evaluating Elearning Applications ? A Case Study

The paper describes a usability approach to evaluating E-Learning applications. In the context of an evaluation study of a computer based training on “leadership in public administration” accomplished for the ministry of the Interior of North-Rhine-Westphalia (Germany), a multimethod approach has been developed that combines traditional methods of empirical research, for example questionnaire and guided interview, with approaches from usability engineering, like eyetracking and annotation of multimodal behaviour. Although this approach is quite complex it reveals to be very helpful in improving the quality of an e-learning application. The paper points out that with this approach we are able to identify problems in using the software that would not have been recognized with a less complex methodology relying on a limited set of data. Some exemplary results form our usability setup demonstrates the potential of this procedure for formative evaluation.